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ABSTRACT 


In this thesis, NOGAPS and COAMPS model data are fused with Alexander 
(2005) algorithm to determine its usefulness in enhaneing satellite-based aireraft ieing 
analysis. This is a follow on to Cooper (2006) researeh where MM5 and ETA were used. 
Using historieal NOGAPS and COAMPS data (T, Td and RH) aceessed from the 
GODAE server, several storms from 2004 were fused with available MODIS imagery 
from the same storms to produee an enhaneed ieing produet. Pilot reports (PIREPS) were 
used as a validation tool to determine where ieing was taking plaee during the storms 
analyzed. A eomparison was made between the MODIS-based ieing potential and the 
model-based ieing potential. The two ieing potentials were fused together to produee an 
enhaneed ieing produet. Statistieal analysis using ROC eurves was performed on the 
various eombinations to determine whieh produet eombination gave the best results. Two 
different available Tmap (Alexander and CIP) were used and had mixed results. Contrary 
to what Cooper (2006) found where weighting RH and the Alexander Tmap produeed the 
best results; this study found that equal weighting of T and RH and the CIP Tmap 
produeed the same or better results than weighting RH. This study also found that 
NOGAPS eombined with the MODIS algorithm provide the best ieing potential results. 
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I. 


INTRODUCTION 


A. BACKGROUND 

The military is becoming more dependent on remotely operated aircraft, known as 
unmanned aerial vehicles (UAV), and precision guided land attack missiles. While this 
minimizes the risk to human life, there are still risks that are shared with manned aircraft 
such as in-flight icing. Icing is important because “[i]t destroys the smooth flow of air, 
increasing drag while decreasing the ability of the airfoil to create lift” (AOPA 2002). Ice 
that has accreted on a wing is depicted in Figure I. 



Figure 1. Ice accreted on wing from (From Hearn 2007) 
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The disruption of air over the airfoil ean eventually lead to a stall and possible crash. It is 
impossible to avoid icing situations unless all flight operations are cancelled, which is 
unreasonable in both civilian and military operations, but icing can be mitigated with 
good forecasting and aircraft mounted deicing equipment. 

B. MOTIVATION 

Aircraft mounted deicing equipment can not be relied upon in extreme icing 
conditions and in some cases can not be operated continuously. Therefore, it is extremely 
important that the pilot/UAV operator know about potential icing situations so they can 
insure their equipment is operating correctly and possibly avoid the potential icing areas. 
This is a daunting task given that some UAV missions exceed 24 hours in length and 
some may even approach 48 hours. The possibility of a two day mission is where the 
forecaster must have the best available tools at their disposable. There has already been a 
loss of several RQ-1 Predators’ (UAV) due to icing (Kilian 2001). Land attack missiles 
and other precision munitions can be susceptible to icing and are often launched over 
areas where there is limited meteorological data. Lack of meteorological data and 
extended forecasts require that a forecaster use more than thumb rules for icing 
calculations. 

Forecasters in the United States, both civilian and military, seem to have a good 
handle on forecasting icing potential. Part of their success stems from the detailed 
meteorological observations available including remote sensing from radars and 
satellites. The current icing potential (CIP) product from the National Weather Service 
(NWS) uses a combination of products such as model output, radar, satellite and pilot 
reports to produce the best available current icing potential. Unfortunately, the areas 
where the military operates typically do not have meteorological radar coverage or pilot 
reports. This means that the forecasters must rely on model output and satellite data. 
Currently, most forecasters rely on the model temperature (T) and relative humidity (RH) 
fields to determine icing potential. This method results in large areas of potential icing 
where actual icing may not be occurring. While this provides the greatest safety cushion, 
it limits the planning of military missions and possibly jeopardizes time-sensitive 
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operations. Alexander (2005) showed that MODIS imagery eould be used to predict icing 
potential and Cooper (2006) showed that the MODIS icing potential could be fused with 
model data, specifically ETA and MM5, to produce a better CIP product. This fused icing 
product would be very helpful to all military planners. Unfortunately, the military 
services, specifically the United States Air Force (USAF) and United States Navy (USN), 
use their own models due to their different operational environments. The USN conducts 
many of their operations over water and relies on the NOGAPS and CO AMPS models so 
a different study is needed to see if the same benefits can be obtained using NOGAPS 
and COAMPS that Cooper (2006) found for the MM5. 

C. PURPOSE 

The purpose of this research is to improve upon the Alexander (2005) MODIS 
9-channel fuzzy logic icing algorithm using NOGAPS and COAMPS model data. 
Alexander (2005) showed that the MODIS icing algorithm did a better job than the 
current GOES icing algorithm mainly due to the increase in available satellite channels. 
Cooper (2006) fused the MODIS icing algorithm output with model data to produce a 
better CIP product than the MODIS icing algorithm alone. This research will be a follow 
on to Cooper (2006) research using NOGAPS and COAMPS model data and determine if 
the USN’s operational models improve upon Alexander’s MODIS icing algorithm. 

D. THESIS PLAN 

Using historical pilot reports (PIREPS) some major icing events were identified 
from January 2004. NOGAPS and COAMPS historical data from the Global Ocean Data 
Assimilation Experiment (GODAE) were downloaded in GRIB format and decoded. This 
data was fused with MODIS imagery that has been processed with Alexander (2005) 
MODIS icing algorithm. Using the historical PIREPS, statistics were produced to 
determine what value if any is added from the model data to the MODIS icing algorithm. 
It is possible that the value added will change with various meteorological situations and 
it must be determined if having a forecaster in the loop (FITE) will be needed to analyze 
the output vice relying on straight automation of the process. The theory will be 
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discussed in Chapter II. Chapter III will eover the data eolleetion and verifieation 
proeess. In Chapter IV the results will be revealed followed by eonelusion and 
reeommendations in Chapter V. 
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II. THEORY 


A. MODIS BASED ICING POTENTIAL ALGORITHM 

Using nine of the 36 MODIS channels Alexander (2005) constructed an algorithm 
that used fuzzy logic over a series of twelve tests to produce an icing potential. The 
twelve tests were broken up into four groups and listed in Table 1 below. The tests are 
thoroughly explained in Alexander (2005), to a lesser extent in Cooper (2006) and will 
only be briefly explained here. 


Table 1. MODIS Icing Test (From Alexander 2005) 


Test Group 

Test (no units unless noted) 

Icing Reflectance Thresholds 

I 

0.65 pm Reflectance (POl) 

Min <0.10 Max >0.25 

I 

1.63 pm Reflectance (P06) 

>0.5 

I 

2.10 pm Reflectance (P07) 

>0.4 

I 

3.90 pm Reflectance (P22) 

>0.06 

I 

Cirrus Reflectance (P26) 

<0.08 

II 

1.63 pm Ratio (P61) 

Min < 0.2 Max >0 .9 

II 

2.10pmRatio (P71) 

Min <0.15 Max > 0.65 

III 

Temperature (C) (P31) 

Min > 0 & < -40 Max=-10 

IV 

3.9-11 pmBTD (C) (BTDl) 

>10 (Day) 

IV 

8-llpmBTD(C) (BTD2) 

Min > 3 Max< -2 

IV 

11-12 pmBTD (C) (BTD3) 

<-0.5 &> 4.5 

IV 

Trispectral BTD (C)(BTD4) 

Same as 8-11 pm BTD 
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1 . 


Group I Reflectance Tests 


Reflectance is simply a measurement of the amount of energy that is received at 
the satellite after being reflected by a surface. Alexander uses Channels 1 (O.hSpm), 6 
(l.hpm), 7 (l.lpm), 22 (3.9)j,m) and 26 (l.SSpm), which due to their wavelength interact 
with the cloud particles, to help separate clouds from the background land or ocean. 
Channels 6, 7 and 26 can be used to determine cloud phase because of their scattering 
property differences. Channel 26 whose wavelength is strongly absorbed by water vapor 
can look for ice clouds where the temperature is so cold there is little or no chance of 
icing. 


a. POl Reflectance Test 

Using MODIS channel 1 clouds can be distinguished from ground or 
ocean, but this is more difficult if the ground is desert or snow/ice covered. Thick clouds 
will have a very high reflectance and the icing probability as a function of channel 1 
reflectance is shown in Figure 2. 
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b. P06 Reflectance Test 


Using MODIS channel 6 the phase of the eloud ean be determined. While 
the real indiees of refraetion are nearly equal at the 1.63 pm wavelength looking at Figure 
3 below one ean see that imaginary indiees of refraetion are signifieantly different for iee 
and water. It is this differenee that allows for phase determination. Figure 4 illustrates the 
ieing potential as a funetion of the ehannel 6 refleetanee. 
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c. 


P07 Reflectance Test 


The P07 reflectance test is very similar to the P06 test except the 
imaginary index of refraction between water and ice is slightly less. Figure 5 shows icing 
potential as a function of channel 7 reflectance. 



2005) 


d. P22 Reflectance Test 

The P22 reflectance test takes place in the transition zone (3.9 pm 
wavelength) between the satellite sensing incoming solar radiation and outgoing 
terrestrial radiation. Because water clouds reflect more energy than ice clouds once again 
phase can be determined. Figure 6 shows icing potential as a function of channel 22 
reflectance. 
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Figure 6. P22 icing potential vs. CH 22 reflectance percentage (From Alexander 

2005) 


e. P26 Reflectance Test 

This test differs from the previous tests because it is not determining 
where the icing potential is high, but where there is no or very little icing potential. 
Figure 7 illustrates icing potential as a function of channel 26 reflectance. 


10 



100 



>N 


05 

o 

D. 

O) 

c 

o 


0 I-^ 

0 8 


Channel 26 Reflectance (%) 


100 


Figure 7. P26 icing potential vs. CH 26 reflectance percentage (From Alexander 

2005) 


2. Group II Reflectance Tests 

The Group II reflectance tests are ratio tests that are better suited for determining 
phase than the Group I tests alone. Channels 1, 6 and 7 were used from the Group I tests 
and two ratios were determined; channel 6 to channel 1 (P61) and channel 7 to channel 1 
(P71). Water clouds will have a higher reflectance than ice clouds and since supercooled 
liquid water (SLW) is a major factor in icing and would exhibit a high ratio for channels 
sensitive to cloud phase. As shown in Figure 8 for P61 and Figure 9 for P71 icing 
probability increases sharply for Channel 6 at 60% reflectance ratio and is 100% icing 
probability when the ratio is 90% and greater. Icing probability follows a similar pattern 
for Channel 7, but the sharp increase is at 35% and is 100% icing probability when the 
ratio is 65%. 
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Channel 6 vs Channel 1 Reflectance Ratio (%) 



P71 icing potential vs. P71 ratio (From Alexander 2005) 




3. Group III Brightness Temperature Test (P31) 


MODIS channel 31 covers the 11 pm wavelength and at this wavelength clouds 
are both good absorbers and good emitters. The earth’s radiation at this wavelength is 
absorbed and emitted by the elouds. Beeause the satellite is sensing the top few meters of 
the elouds, the eloud top temperature ean be aeeurately aseertained and the remaining 
eloud temperature ean be assumed to be warmer. Figure 10 illustrates ieing probability as 
a funetion of that brightness temperature. 



4. Group IV Brightness Temperature Difference Tests 

The Group IV tests are used for phase differentiation within the clouds by using 
the difference in imaginary index of refraetion for water and iee from one wavelength to 
the next. The brightness temperature will change as the wavelength changes and that 
ehange ean exploited for the differenees between water and iee. 
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a. BTDl Brightness Temperature Difference Test 

During daylight hours channel 22 will have a higher brightness 
temperature than ehannel 31 beeause at the ehannel 22 wavelength, energy is refleeted 
and emitted by the elouds (Cooper 2006). If it is a water eloud the differenee will even be 
greater sinee water elouds have a higher refleetanee at the ehannel 22 wavelength. Figure 
11 shows that as the differenee between ehannel 22 and ehannel 31 beeomes larger the 
ieing potential also beeomes greater. 



Figure 11. BTDl ieing potential vs. CH 22-31 brightness temperature differenee 
(From Alexander 2005) 


b. BTDl Brightness Temperature Difference Test 

The BTD2 brightness temperature differenee test is slightly different from 
the previous tests sinee it ean produee a negative value. The test is based on taking the 
brightness temperature differenee between ehannels 29 and 31. The differenee ean 
produee a negative value sinee the absorption ooeffieient for water and iee are the same 
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for channel 29, but increases in channel 31 for ice. Water clouds will produce a negative 
value and ice clouds a positive value. The icing probability as a function of the difference 
is shown in Figure 12. 


100 



Channel 29 - 31 BTD (°C) 


Figure 12. BTD2 icing potential vs. CH 29-31 brightness temperature difference 
(From Alexander 2005) 


c. BTD3 Brightness Temperature Difference Test 

BTD3 brightness temperature difference is somewhat similar to BTD 2, 
but its main purpose is to identify cirrus clouds. Figure 13 shows the band, 0.8-1.5, where 
cirrus would be likely. 
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Figure 13. BTD3 icing potential vs. CH 31-32 brightness temperature difference 
(From Alexander 2005) 


d. BTD4 Trispectral Brightness Temperature Difference Test 

The trispectral brightness temperature difference test is the difference 
between BTD2 and BTD3. Due to their temperature difference relations, if it is a water 
cloud BTD2 would be negative and BTD3 would be positive resulting in a large negative 
number. If there is good mixture of water and ice clouds the value will be closer to zero. 
If it is strictly an ice cloud the value will be positive. Figure 14 shows the icing potential 
as a function of the BTD2 and BTD3 difference. 
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3 

Trispectral [(29-31) - (31-32)] BTD (°C) 

Figure 14. BTD4 icing potential vs. trispectral brightness temperature difference 
(From Alexander 2005) 

5. Final MODIS Algorithm Test 

The final MODIS algorithm test is a simple mathematical function that takes the 
highest value for each pixel of each group and multiplies the four groups together and 
then the fourth root is taken to determine the final icing potential. A flowchart of the 
algorithm is shown in Figure 15. 
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Figure 15. Flowchart for the final MODIS algorithm test (From Cooper 2006) 

B. MODEL BASED ICING POTENTIAL DETERMINATION 

Model determination of icing in its simplest form involves answering two 
questions: Where are the clouds and where in the clouds is it conducive to icing? The 
simplest two model parameters that answer those two questions are temperature and 
relative humidity. There have been several studies done to answer at what RH are cloud 
formations likely. While the top number of 100% is a given it is the lower threshold 
where the debate lies. Looking at Figure 16 one can see that icing can occur as low as 
30% RH. Relative Humidity of 60% and greater is a common assumption for cloud 
formation. Using the RH based Interest Map in Figure 16 allows the use of fuzzy logic 
where the higher RH’s are weighted heavier than the lower ones. 
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CIP PHmsp based on Bernstein et al (20115) 



Figure 16. Icing potential as a function of relative humidity (From Bernstein et al. 
2005) 


The second piece needed for icing potential is temperature. The upper limit of OC 
is a given, but the lower limit and where icing peaks is open for discussion. For the lower 
limit, it is known that between SLW begins to instantaneously freeze between -25 C and 
-40C there is no longer an icing threat. The two T Interest Maps used in this study and 
Cooper’s similar study, shown in Figure 17, show different peaks and different tails for 
the lower limits. Alexander’s T Interest Map has a distinct peak, but the CIP developed 
by Bernstein et al. has a peak over a range of values which gives a higher icing 
probability over a broader range of temperatures. The results section will illustrate which 
one was more successful in this study. 
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Aitctafl Icing Potential Percentage 



Using the T and RH Interest Maps together an ieing probability ean be 
determined. 


AitianUtift.iP fmypbvitQ ft 4 
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Figure 17. T Interest Map: Icing potential as a function of temperature (black = From 
Alexander 2005 and blue = From Bernstein et al. 2005) 
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III. PROCEDURE 


The original idea for this study was to collect model data from storms transiting 
the northeastern United States during late spring into early summer. Unfortunately the 
storms did not produce adequate icing for this study. However, historical NOGAPS and 
COAMPS data stored on the GODAE server at http://www.usgodae.org/cgi- 
bin/datalist.pl?generate=summarv was extremely valuable in completing this study. Data 
from early 2004 was used since it contained some significant icing events and the most 
complete data sets. The northeastern United States was chosen due to the amount of 
PIREPS available in that area and is also consistent with the area studied in Alexander 
(2005) and Cooper (2006). MODIS imagery for the same the same time period was 
obtained from the National Aeronautics and Space Administration (NASA) website at 
http://ladsweb.nascom.nasa.gov/data/search.html . PIREPS were obtained from the 
National Climatic Data Center (NCDC) website at 
http://hurricane.ncdc.noaa.gov/pls/plhas/has.dsselect . 

A. NOGAPS AND COAMPS MODEL INVESTIGATION 

Fleet Numerical Meteorological and Oceanography Center (FNMOC) model data 
stored on the GODAE server was downloaded in gridded binary (GRIB) format and 
decoded using the commercially available Integrated Data Viewer (IDV) by Unidata. 
Using the IDV Temperature, Dewpoint and RH data from the closest level to the 
available PIREP (typically 850mb) was retrieved and put into an Excel spreadsheet for 
analysis. The forecast model run that preceded the PIREP time was used, and the data 
came from the forecast hour closest to the PIREP (i.e., PIREP is 1310 so the I2Z model 
run would be used and the 1300 data would be used). Figure 18 shows an example of the 
IDV output of 850mb temperatures over North America. 
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Figure 18. IDV display of 850mb temperatures over North America 


Obtaining data from the IDV is quite simple by use of a data probe that allows the 
user to find the exact data for any latitude and longitude. If the PIREP reported level fell 
between two model levels, then the two levels were investigated and interpolated if 
needed. Since RFl was one of the model output parameters it was not necessary to 
calculate RH from the temperature and dewpoint. However RH was calculated from 
model temperature and dewpoint to insure the model output consistency. 

B. MODIS LEVEL 1 DATA INVESTIGATION 

Using the MODIS algorithm developed by Alexander (2005), MODIS level-1 
data was decoded and compared to the corresponding PIREPS. PIREPS from four hours 
on either side of the MODIS image were used for comparison. This is a slight change 
from Cooper (2006) who used only three hours on either side. This change was necessary 
due to receiving PIREPS from the archive versus real time. There are significantly more 
PIREPS available in real time that makes it to the archive. In order to have more PIREPS 
available, the observational time on either side of the PIREP was extended slightly. When 
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the MODIS image is eompared to the PIREPS pixels within 25 miles of the PIREP are 
examined to determine the mean and median values within that area. The maximum of 
either the mean or median is used to determine the MODIS ieing potential (MODISpot) 
(Cooper 2006). 

C. TOTAL ICING POTENTIAL CALCULATIONS 

1. Model Icing Potential Assignment 

Temperature and relative humidity are the only two parameters needed for the 
simplistie model ieing potential assignment. Both parameters are available in the model 
output. Ieing potential due to model RH is assigned based on the RH Interest Map (Figure 
16). Ieing potential due to model T is assigned based on the use of Alexander or CIP T 
Interest Map (Figure 17). When the MATLAB eode is run the user has a ehoiee on whieh 
Tmap is used. For the purpose of this study both Tmap were used for eomparison. 

2. Five Test Calculations 

Five test ealeulations using both Tmap curves were eondueted resulting in 10 
overall icing potentials. These tests are the same that Cooper (2006) used with the 
exception of the 1.5 hr difference model forecast which resulted in eight tests in that 
study. COAMPS is shown, but the same ealeulations were also used for the NOGAPS 
data. Cooper (2006) did the same for ETA in his study but did not inelude the ETA 
results. These tests are neeessary to determine whieh faetor, RH or T, is more important 
in the ieing potential ealculation and to determine how mueh the model data should be 
weighted eompared to the MODIS data. Using both Tmap curves determines whieh Tmap 
provides the best results. Eaeh variable and ealeulation is deseribed below and shown in 
Table 2. 


23 



a. Relative Humidity 

The RH formula, shown in Table 2, for ealeulating RH was used in 
Cooper (2006), but was not needed in this study beeause it was given in the model output. 
It was used as a validation tool to insure the Td model output and RH were in agreement. 

b. COAMPSl 

COAMPS 1 gives equal weighting to Tpot and RHpot whieh are multiplied 
together and square root taken. The square root result is then multiplied by the MODISpot 
and the square root of that gives the ieing potential for COAMPSl. 

c. COAMPSl 

COAMPS2 gives more weighting to Tpot by multiplying it by itself and 
then multiplying RHpot and taking the eube root. The cube root result is then multiplied 
by the MODISpot and the square root of that gives the icing potential for COAMPS2. 

d. COAMPS3 

COAMPS3 is the same as COAMPS2 except the extra weighting is given 
to the RHpot by multiplying it by itself and then multiplying Tpot and taking the cube root. 
The cube root result is then multiplied by the MODISpot and the square root of that gives 
the icing potential for COAMPS3. 

e. COAMPS4 

COAMPS4 simply weights MODISpot by 60% and COAMPS3 by 40%. 

/. COAMPS5 

COAMPS5 takes COAMSP3 result multiplied by the MODISpot and the 
square root of that is the icing potential of COAMPS5. Cooper (2006) used this 
calculation due to the favorable results of the similar COAMPS3 calculation. 
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Table 2. COAMPS Model leing Calculations 


Variable 

Eormula 

Relative Humidity (RH) 

R//«100(^-^^—Q- (tyjational Weather Service 

112 + 0.9r 


2005) 

Icing Potential Test 1 using 

T and Td from closest 

forecast hour (COAMPS 1) 

COAMPSl = )* MODlS„„, _ 

^ pot pot) pot (Cooper 2006) 

Icing Potential Test 2 

(COAMPS2) 

COAMPS 2 = ) • MOD/5^„ (Cooper 

2006) 

Icing Potential Test 3 

(COAMPS3) 

COAMPSL = •RH^,*RH^, yMODlS^, (Cooper 

2006) 

Icing Potential Test 4 

(COAMPS4) 

COAMPS A = 0.6 * MODISp^, + 0.4 * COAMPSL 

Icing Potential Test 5 

(COAMPS5) 

COAMPS5 = ^COAMPS3 * MODIS^^, 


D. VERIFICATION 


To provide a direct comparison with Cooper (2006), a similar verification system 
was used that followed from Alexander (2005). If the PIREP location had a 0.4 value or 
greater it was considered high icing potential while less than 0.4 was considered low 
icing potential. Using these potential values they were compared with the actual positive 
and negative PIREPS to determine which ones were correctly identified. Alexander 
(2005) and Cooper (2006) used a 0.5 threshold, but after analyzing data for this study a 
threshold of 0.4 was determined to be a better fit for the data in this study. 
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From the above results, the probability of deteetion (POD), also known as 
sensitivity, ean be ealeulated by taking the total number of eorreetly identified positive 
PIREPS divided by the total number of positive PIREPS. The probability of detection 
null (PODno), also known as specificity, is calculated in a similar manner by taking the 
number of correctly identified negative PIREPS divided by the total number of negative 
PIREPS (Cooper 2006). The results are discussed in the next chapter. 
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IV. RESULTS 


Cooper (2006) used receiver operating characteristic curves (ROC), which are 
simply a graphical plot of sensitivity versus one minus specificity, for presentation of his 
MODIS and MM5 data. For comparison, similar data styles and ROC curves are 
provided in this study. ROC curves are good for data that can be identified as true or 
false, which is the case for identifying positive or negative PIREPS. Icing probabilities 
from the MODIS icing algorithm and the model icing algorithm are compared to the 
historic PIREPS to determine whether the icing is correctly identified. An icing 
probability of 0.4 or greater compared to a positive PIREP will yield a true result while 
an icing probability of 0.4 or greater compared to a negative PIREP will yield a false 
positive. The inverse is also true; an icing probability of less than 0.4 compared to a 
positive PIREP will yield a false negative while an icing probability of less than 0.4 
compared to a negative PIREP will yield a true result. Accuracy is calculated by taking 
the number of positive or negative PIREPS identified correctly divided by the total 
number of PIREPS. Sensitivity is the number of correctly identified positive PIREPS 
divided by the total number of positive PIREPS. Specificity is the number of correctly 
identified negative PIREPS divided by the total number of negative PIREPS. 

A. ALEXANDER Tmap 

As stated earlier, there are two distinctive Tmap maps available for calculating 
icing probability. The first one used for model data output is the Alexander Tmap that is 
shown in Figure 17 as a solid black line. Using a commercially available ROC curve 
program, irocfit.org . the MODIS and model icing probability data was input along with 
the PIREPS and the resulting data is shown in Table 3. Along with the accuracy, 
sensitivity and specificity discussed above, the empirical ROC area is the area below and 
to the right of the ROC curves which means the curves with the highest area should 
provide the best results, but that is not always the case. If the data has a lot more of either 
positive or negative cases the data may be artificially skewed. This is the case for the data 
in this study because of the lack of available negative PIREPS. The associated ROC 
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curves from Table 3 are shown in Figure 19. The best possible ROC eurve would be a 
straight line from 0 on the x-axis to the 1 on the y-axis. This means that curves that are 
tilted eloser to the y-axis are generally better. 


Table 3. _ Caleulation Results for PIREPS using Alexander Tmap 


Calculation 

Accuracy (%) 

Sensitivity (%) 

Specificity (%) 

Empirical 

ROC Area 

MODIS 

58.8 

57.1 

66.7 

0.476 

CO AMPS 1 

64.7 

67.9 

50.0 

0.568 

COAMPS2 

61.8 

64.3 

50.0 

0.554 

COAMPS3 

73.5 

82.1 

33.3 

0.580 

COAMPS4 

58.8 

57.1 

66.7 

0.637 

COAMPS5 

58.8 

57.1 

66.7 

0.637 

NOGAPS 1 

82.4 

89.3 

50.0 

0.613 

NOGAPS2 

79.4 

85.7 

50.0 

0.583 

NOGAPS3 

82.4 

89.3 

50.0 

0.619 

NOGAPS4 

55.9 

57.1 

50.0 

0.563 

NOGAPS5 

58.8 

57.1 

66.7 

0.649 


1. ROC Curves Using Alexander Tmap 

Looking at Table 3 above and Figure 19 below, one ean see that the best ROC 
curve is NOGAPS5 whieh is the furthest to the left and has the highest ROC area. While 
seientiflcally NOGAPS5 is the best result from the ROC process, the laek of negative 
PIREPS, whieh was also a problem for Alexander (2005) and Cooper (2006), artifieially 
skews the results. Since the data is skewed, the data should be examined in more detail in 
the tabular format. 
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There were six times as many positive PIREPS compared to negative PIREPS so 
one of the most significant results is the sensitivity. A high sensitivity means the positive 
PIREPS were properly identified. The MODIS calculation had one of the lowest 
sensitivities coming in just under 60%. The COAMPS 1 calculation that weights 
temperature and humidity equally increased the sensitivity to just under than 70%. The 
T-weighted COAMPS2 reduced to a sensitivity of around 65%. The COAMPS3 
sensitivity which weights RH more gives the highest sensitivity for the COAMPS group 
at just over 80%. COAMPS4 and COAMPS5 which use different weighting of the 
COAMPS3 field are very similar to the MODIS field by itself with a sensitivity value just 
under 60%. The NOGAPS fields had a similar trend with NOGAPS 1 producing a 
sensitivity of just under 90%. NOGAPS2, similar to COAMPS2, reduced slightly to a 
sensitivity of around 85%. NOGAPS3 had one of the highest sensitivities for the 
NOGAPS group, just like COAMPS3 for the COAMPS group, with a sensitivity value 
just below 90%. NOGAPS4 and NOGAPS5 decrease significantly to a sensitivity value 
just below 60%. 

Based on high sensitivity values, NOGAPS 1, NOGAPS3, NOGAPS2 and 
COAMPS3 provide the best results. Heavier weighting of RH also produced the best 
results for Cooper (2006). Even though the number of negative PIREPS makes them 
statistically insignificant, the NOGAPS group identified at least 50% or more negative 
PIREPS correctly. The COAMPS group did the same with the exception of COAMPS3 
which only correctly identified 33% of the negative PIREPS. This gives the edge for 
accurately predicting icing to the NOGAPS group, specifically the equally weighted 
NOGAPS 1 and the RH-weighted NOGAPS3. 
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Alexander T Map ROC 



— MODIS 
— COAMPS1 
—A— COAMPS2 
COAMPS3 
— COAMPS4 
— COAMPS5 
—I— NOGAPS1 

- NOGAPS2 

NOGAPS3 

NOGAPS4 

NOGAPS5 


Figure 19. ROC curves for MODIS and model icing potentials using 
Alexander Tmap 


B. CIP Tmap 

The CIP Tmap is shown as the solid blue line in Figure 17. The CIP Tmap assigns a 
higher icing probability over a broader range of temperatures and the ROC curve results 
are shown in Table 4 below. Contrary to the data results in Cooper (2006), the accuracies 
and sensitivities for the CIP Tmap were either better or at least the same as the Alexander 
Tmap- The associated ROC curves for the CIP Tmap are shown in Figure 20 below. 
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Table 4. Calculation Results 1 

fir PIREPS using CIP Tmap 


Calculation 

Accuracy (%) 

Sensitivity (%) 

Specificity (%) 

Empirical ROC 

Area 

MODIS 

58.8 

57.1 

66.7 

0.476 

CO AMPS I 

91.2 

96.4 

66.7 

0.696 

COAMPS2 

73.5 

75.0 

66.7 

0.667 

COAMPS3 

85.3 

92.9 

50.0 

0.667 

COAMPS4 

58.8 

57.1 

66.7 

0.625 

COAMPS5 

58.8 

57.1 

66.7 

0.637 

NOGAPS I 

91.2 

96.4 

66.7 

0.679 

NOGAPS2 

91.2 

96.4 

66.7 

0.679 

NOGAPS3 

88.2 

96.4 

50.0 

0.646 

NOGAPS4 

58.8 

57.1 

66.7 

0.637 

NOGAPS5 

58.8 

57.1 

66.7 

0.649 


1. ROC Curves Using CIP Tmap 

Looking at table 4 above and Figure 20 below one can see the best ROC curve is 
NOGAPS 1. Unlike the Alexander Tmap where the results were skewed by the lack of 
negative PIREPS the lack of negative PIREPS did not have as great of an effect on the 
CIP Tmap results. The MODIS calculation did not change from the Alexander Tmap and has 
a sensitivity of just under 60%. CO AMPS 1 which has the highest ROC area also has the 
highest sensitivity with a value just over 95%. The COAMPS2 calculation reduces about 
20% and has a sensitivity value around 75%. COAMPS3 increased about 20% and has a 
sensitivity of just over 90%. COAMPS4 and COAMPS5 share sensitivities with the 
MODIS value of just under 60%. The NOGAPS fields start off strongly with NOGAPS 1, 
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N0GAPS2 and N0GAPS3 each having sensitivities of just over 95%. NOGAPS4 and 
NOGAPS5, just like their equals in the CO AMPS group, share the MODIS sensitivity of 
just under 60%. 

COAMPSl, NOGAPS 1, NOGAPS2 and NOGAPS3 produced the best results 
respectively. Unlike the Alexander Tmap calculations where the specificities dropped 
below 50% none of the CIP Tmap calculations reach that level. All of the calculations 
correctly identify 2 out of 3 negative PIREPS with the exception of COAMPS3 and 
NOGAPS3 which only reach the 50% level. The calculation for COAMPSl and 
NOGAPS 1 weights T and RH equally which is significant since Cooper (2006) found 
weighting RH more to be crucial in that study and was an important factor for the 
Alexander Tmap results in this study. This shows that where the model output T lies on the 
Tmap plays a very important role in the results. If several of the T model data points lie in 
the 0 to -10 region as they did for this study that can have significant impact on the 
results when they are weighted heavier over a broader region as is the case for the CIP 
Tmap- Just like the Alexander Tmap the edge for correctly identifying icing goes to the 
NOGAPS group with the same score for the equally weighted NOGAPS 1 and the 
T-weighed NOGAPS2. 
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CiP T Map ROC 
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NOGAPS5 


Figure 20. ROC curves for MODIS and model icing potentials using CIP 
Tmap 
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V. CONCLUSION AND RECOMMENDATIONS 


A. CONCLUSION 

In a perfect world the results of this study would have meshed perfectly with the 
findings of Cooper (2006). Unfortunately, this is not the case and this study revealed 
different findings. The first major difference is that the threshold for detecting icing was 
lowered to 40% vice the 50% used by Alexander (2005) and Cooper (2006). Several 
thresholds were evaluated and 40% clearly produced the best results for both the MODIS 
and model icing algorithms. Alexander (2005) and Cooper (2006) suggest looking at 
some variations in the icing threshold, but there is insufficient explanation for why 50% 
was chosen. Due to this slight variation between theses, this threshold should be 
examined for all models and the MODIS icing algorithm. The second major difference 
compared to Cooper (2006) is the better performance of the CIP Tmap compared to the 
Alexander Tmap- This can be attributed to many of the temperatures being in the range 
where the CIP Tmap has a broad range of high icing potentials. The third difference was 
related to the weighting of the T and RH. In Cooper (2006), weighting RH did a better 
job of identifying the icing areas. While that was the case to some extent with the 
Alexander Tmap and CIP Tmap, in this study the CIP Tmap equal weighting of T and RH 
produced the best results for the COAMPS group and one of the best for the NOGAPS 
group. 

NOGAPS beat out CO AMPS for both Tmap test groups, and this is not a surprise 
since icing is predominantly a synoptic scale event, but it should be noted that mesoscale 
factors such as very mountainous terrain like what is experienced in Afghanistan could 
have a significant impact on icing development. Another shortcoming of this study is the 
lack of available PIREPS especially negative PIREPS. While there were sufficient 
positive PIREPS to show some statistical significance the lack of negative PIREPS 
skewed the ROC analysis. While it can be seen that the COAMPS and NOGAPS model 
data enhance the detection of icing when combined with the MODIS data, it is not clear 
on how discriminatory the product is. Without being able to verify that the algorithm is 
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adequately identifying non-icing areas it is difficult to call it a total success, yet I do 
believe there is some operational usefulness in this product. Even in its limited form it 
could be another tool for an operational commander or forecaster to make an informed 
icing potential forecast. 

The armed forces’ method of hatching out an area on a map and saying there is 
going to be icing within the hatched area is not a very useful operational tool. An 
operational commander trying to make an informed decision about whether the mission 
can succeed or not should be supplied with probabilistic information vice an either on or 
off deterministic approach. This combined model and MODIS algorithm product is a 
stepping stone in developing such a probabilistic product. Already in development is an 
ensemble model that is capable of a probabilistic icing product. With this new ensemble 
forecast, combined with the MODIS algorithm the icing forecast product can be greatly 
improved. 

B. RECOMMENDATIONS 

The first recommendation would be to continue this study based on the new 
ensemble model that is being developed. A model capable of producing a probabilistic 
product along with the MODIS algorithm that can produce a probabilistic output would 
be very beneficial to military planners. It would not be a far reach to continue this study 
on volumetric data vice just point sources. Integrating the data both horizontally and 
vertically could produce a 3-D picture to help the forecaster or military planner to 
visualize the situation. With a visualization tool it might be easier to see what is needed 
to mitigate the icing potential. Developing a 3-D picture with model data will not be 
difficult, but some complexities arise with the MODIS algorithm due to the limitations of 
looking through the entire cloud. Another limitation to this approach is verifying the 
extent of icing or lack of icing throughout the volume since PIREPS only provide point 
source information. This could be overcome in a detailed study with an available airplane 
to verily icing conditions. Another recommendation is to further identify where a 
forecaster in the loop (FITE) needs to be placed in the process. Where to place the FITE 
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might change upon changing weather conditions or terrain, but needs to be identified in 
detail so the FITL is effectively utilized. 

With the progression of theses from Alexander (2005), Cooper (2006) and this 
study there is sufficient background work to attempt a major study that would combine 
all the previous models used along with the new ensemble model and the MODIS 
algorithm to determine which model performs best under several conditions. There are 
several obstacles to be overcome such as increasing the availability of PIREPS to verify 
icing and setting the study up over a sufficient length of time to cover multiple icing 
situations. Mesoscale effects such as mountainous areas need to be looked at to see if that 
has a significant impact on the icing product. Due to the complexity of this problem, it is 
highly recommended that a PhD student complete this study. 
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